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DETAILED ACTION 
Claim Objections 

1 . Claim 1 objected to because of the following informalities: "real scene" in line 3 
should be "said real scene". Appropriate correction is required. 

2. Claim 1 objected to because of the following informalities: "desired viewpoint" in 
line 7 should be "said desired viewpoint". Appropriate correction is required. 

3. Claim 1 objected to because of the following informalities: "selected object" in line 
1 1 should be "said selected object". Appropriate correction is required. 

4. Claim 2 objected to because of the following informalities: "real image" in line 2 
should be "a real image". Appropriate correction is required. 

Claim Rejections - 35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1.02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in tfiis or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
states. 

6. Claim 1 -1 0, 1 2-30,32-35 are rejected under 35 U.S.C. 1 02(b) as being 
anticipated by Moezzi (U.S. Patent Number 5,850,352). 

As to claim 1 , Moezzi discloses, a method for generating a desired view of a real 
scene from a selected desired viewpoint, (example, selection of desired view, real world 
scene, see column 20, lines 30-34, column 9, line 51 and figure 1a-1c) said method 
comprising: obtaining at least one real scene image from one or more cameras 
(example, real-world event, multiple video camera, see column, 39, lines 43-44) said 
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one or more cameras each having a respective real viewpoint (see figure 10) identifying 
selected objects (example, selected object, column 1 , line 39), (example, detect objects 
or identify event, see column 26, lines 15-16, figure 2) in said at least one real scene 
image detemnlning estimates of the positions of the selected objects (example, 
positions, see column 26, line 20, figure 2) selecting a desired viewpoint (see column 
34, lines 49-52, column 20, line 32) based on the relationship (example, relationship, 
column 38, Iine22) of the selected desired viewpoint to the or each real viewpoint, and 
the estimates of the positions of the selected objects, determining positions of the 
selected objects in said desired view of the scene and rendering ( see figure 2) a view 
of the scene from the selected desired viewpoint (see column 20, lines 3-4) wherein at 
least some selected objects are rendered using image data (example, image data, see 
column 5, line 66) from at least one real scene source image ( column 1 , line 42). 

As to claim 2, Moezzi discloses, wherein at least a portion of said rendered view 
is generated without transformation of real images (example virtual images, see column 
1 , lines 34) 

As to claim 3, Moezzi discloses, wherein at least a portion of said rendered view 
is generated using image data from a real scene image (example, real video images 
see column 1 , lines 34) which is not contemporaneous with the image data from which 
said at least some selected objects are rendered (see figure 2). 
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As to claim 4, Moezzi discloses, wherein selected objects are rendered in the 
desired view as projections (example, projection, (see column 38, lines 19-24) of real 
images of those objects obtained from at least one real scene image (see figure 2). 

As to claim 5, Moezzi discloses, wherein said real images of selected objects are 
transformed (example, transformation, column 5, line 65-66 optionally rotated (example, 
sphere, see column 16, lines 19). 

As to claim 6, Moezzi discloses, wherein selected objects are rendered in the 
desired view as projections of real images of those objects oriented perpendicular to the 
real camera optical axis (see figure 2). 

As to claim 7 Moezzi discloses, wherein selected objects are rendered in the 
desired view as projections of real images of those objects oriented perpendicular to the 
selected viewpoint optical axis (see figure 2). 

As to claim 8, Moezzi discloses, wherein selected objects are rendered in the 
desired view as projections of real images, which have been mapped onto 3D surfaces 
(example, mapped into 3-D. see column 43, lines 45-53, figure 4, figure 9b and 9c). 

As to claim 9, Moezzi discloses, wherein said 3D surfaces are generated in 
response to the outline of the real images of said selected objects obtained from at least 
one real scene image (example, outline, selection criteria, see column 32. lines 54-55, 
figure 7). 
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As to claim 10, Moezzi discloses, wherein real images of selected objects are 
obtained from said at least one real scene image by a keying process (example, keying, 
see column 3, lines 55-60). 

As to claim 12, Moezzi discloses, wherein images of selected objects obtained 
from said at least one real scene image is interpolated (see figure 2). 

As to claim 13, Moezzi discloses, wherein a set of real scene images are 
obtained from a plurality of cameras having mutually different viewpoints (example, 
multiple video cameras, different location, See abstract, column, 15, line 40, column 9, 
lines 1 0-1 8 and figure 5). 

As to claim 14, Moezzi discloses, wherein each selected object in the desired 
view is rendered as a projection of a real image of that object extracted from the one of 
said set of real scene images that corresponds to the real viewpoint closest to the 
desired viewpoint (example, extract, received desired view, column 20, lines 36-38). 

As to claim 15, Moezzi discloses, wherein each selected object in the desired 
view is rendered using image data from two or more of said set of real scene images 
(example image data, see column 5, line 66, figure 2). 

As to claim 16, Moezzi discloses, wherein projections of real images are 
projections of real images mapped onto 3D surfaces (example, 3D, texture projection, 
see figure 2, column 43, line 49). 
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As to claim 17, Moezzi discloses, wherein said 3D surfaces are generated from 
the intersections of generalized cones (example, generalization, all points, see column 
31 , Iine58, column 32, line 1 7, figure 9a) of the outline of a selected object viewed from 
different viewpoints (see figure 1 1 ) which generalized cones are the union of visual rays 
from all silhouette points of a selected object (example, silhouettes, see column 37, line 
22, column 28, line 47, figure 4, figure 2). 

As to claim 18, Moezzi discloses, wherein one or more of said real cameras are 
slave cameras, which are automatically controlled based on camera parameters of 
others of said real cameras (example master slave, see column, 38, lines 28-48, figure 
17. 

As to claim 19, Moezzi discloses, wherein said different viewpoints comprises at 
least one elevated viewpoint and at least one low-level viewpoint (example, different 
level, see column 26, lines 63-67, column 27, lines 5-8, and figure 4) 

As to claim 20, Moezzi discloses, wherein images from said elevated viewpoints 
are used to detemiine the position of selected objects (example, position estimation, 
see figure 2) in a scene and/or images from said low-level viewpoints are used to render 
selected objects in the desired view (see figure la). 

As to claim 21, Moezzi discloses, further comprising tracking selected objects in 
one or more sequences of real scene images (example tracking, see column 32, Iine58, 
figure 5 and figure 3). 
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As to claim 22, Moezzi discloses, wherein said object tracking comprises 
obtaining a silhouette of selected objects from a real scene image by keying (example, 
keyed, tracked, silhouette, see column 3. line 57-59, figure 2) and analyzing changes in 
shape or position of the silhouette in successive real scene images (example changes, 
analyzed, track object, positions see column 25, line 42-45, column 26, line 15, 20, 
column 27, line 67). 

As to claim 23, Moezzi discloses, providing a user interface to allow an operator 
to view one or more real scene images (example user interface, see column 26, line 32, 
37-38) and to modify an automatic object tracking process (example, automatically 
tracked, see column 3, line 57). 

As to claim 24, Moezzi discloses, wherein said user interface additionally allows 
an operator to modify the keying of a selected object in one or more real scene images 
(example, keying, see column 3, lines 55-60). 

As to claim 25, Moezzi disclose, apparatus for generating a desired view of a real 
scene from a selected desired viewpoint, (example, selection of desired view point, 
column 20, line 30-34, figure 8b) means for obtaining at least one real scene image 
from one or more cameras (example, viewer interface means, see column20, line30-34) 
the or each camera having a respective real viewpoint (figure 2) means for identifying 
selected objects in said at least one real scene image (example, identifying object, see 
column 26, lines 15-16, figure2) means for determining estimates of the positions of the 
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selected objects (example position estimation, figure 2); means for selecting a desired 
viewpoint (example, object selection, see figure 2) and based on the relationship of the 
selected desired viewpoint to the or each real viewpoint, means for determining 
positions of the selected objects in said desired view of the scene and rendering a view 
of the scene from the selected desired viewpoint wherein at least some selected objects 
are rendered using image data from at least one real scene source image(example, 
model rendering, data, see figure2). 

As to claim 26, Moezzi disclose, a method of rnonitoring a scene for virtual image 
generation (see figure 2, column 23, line 67) said method comprising: obtaining a set of 
real scene images from a plurality of cameras having mutually different viewpoints 
(figure 1a-1c) using image data from at least a first of said real scene Images to derive 
the position of a selected object in the scene (example, object and position selection, 
first scan time video data, see figure 2. column 18, line 12) and using Image data from 
at least a second of said real scene images to render a virtual image of said selected 
object (example model rendering, virtual camera image, second scan time video data, 
see figure 2, column 1 8, line 13). 

As to claim 27, Moezzi disclose, wherein a first subset of real scene images are 
used to derive position (see column 5, Iine60, column 26, lines 12-20) and a second 
subset of real scene images are used for rendering (column 40, lines 20-21). 
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As to claim 28, Moezzi disclose, wherein at least one of said real cameras 
provides an elevated viewpoint, and at least one of said real cameras provides a low- 
level viewpoint and wherein said first subset of images includes images from at least 
one camera having an elevated viewpoint of the scene, and said second subset 
includes image from at least one camera having a low-level viewpoint of the scene (see 
figure la, figure lOa-IOh, example low level, high level see column 25, 35-37). 

As to claim 29, Moezzi disclose, wherein each real camera is located at a 
different lateral orientation around a scene (see figure 1c). 

As to claim 30, Moezzi disclose, a method of controlling a slave camera based 
on the parameters of at least one other camera, said method comprising (example, 
slave see column 38, lines, 35-38) adjusting the parameters of said slave camera to 
point and focus at a desired point based on the camera parameters of at least one of 
said other cameras (column 39, lines 1-13). 

As to claim 32, Moezzi disclose, apparatus for tracking selected objects in a 
scene comprising (example tracking, see figure 5, column 32, line 55) one or more 
cameras arranged to obtain one or more real scene images (see figure 5) image 
processing means for identifying said selected objects in said one or more real scene 
images (example, video data analyzer means, see column 20, lines 23-25) means for 
providing an estimate of the position of said one or more selected objects based on their 
position in the one or more real scene images (see column 26,line 20) a user interface 



Application/Control Number: 10/799,030 Page 10 

Art Unit: 2109 

adapted to allow an operator to view said estimate of the position of selected objects in 
a real scene image said user interface including input means to allow an operator to 
modify said estimate (example user/viewer interface, see column 20, lines 30-34). 

As to claim 33. Moezzi disclose, wherein real scene imagies are obtained from a 
plurality of cameras having different viewpoints (see figure 1 0a-1 Oh); 

As to claim 34, Moezzi disclose, wherein more than one real scene images from 
different viewpoints are displayed simultaneously (example, display see column 15, line 
31, figure 10a-10h) and wherein said estimate is indicated graphically on more than one 
real scene image (see figure 9a-9c). 

As to claim 35. Moezzi disclose, arranged to allow an operator to select those 
cameras from which real scene images (example, selection of real camera or image, 
see column 1 line. 41 ) are used to provide said estimate of location (example different 
location see column 33, line 48). 

Claim Rejections - 35 USC § 103 
7. The following is a quotation of 35 U.S.C. 103(a) which fomns the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the difTerences between the subject nfiatter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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8. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966). that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobvlousness. 

9. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any Inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and Invention dates of each claini that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) ahd potential 35 U.S.C. 102(e). (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

10. Claim 31. 36-38 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Moezzi et al. (US Patent No. 5,850,352) as applied to claim 30 and 32 above, in view of 
Sharir et al. (US Patent. No. 6.380,933). 

As to claim 31, Moezzi does disclose, pan, tilt, zoom and focus parameters (see 
column 32, lines 9-15). However, Moezzi does not specifically disclose wherein all of 
pan, tilt, zooms and focus parameters are controlled automatically. Sharir from the 

same field of endeavor discloses, the pan. tilt, zoom and focus parameters are 
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controlled automatically (see column 2, line 56-58). It would have been obvious to one 
of ordinary skilled in the art to have modified tilt, zoom, and focus parameters as 
presented by Moezzi with automatic control as presented by Sharir. The motivation to 
combine these two references involves automatic measurement of the camera field of 
view (see column 2, lines, 56-58). 

As to claim 36, iSharir disclose, an^anged to allow an operator to indicate the 
position of one or more selected objects in one or more real scene images (see column 
2, line 6). 

As to claim 37, Sharir disclose, arranged to allow an operator to indicate the 
position of one or more selected objects In a first real scene image, and to display an 
estimate of the corresponding position of said one or more objects in at least a second 
real scene image (see column 1, line 59). 

As to claim 38, Sharir disclose, including means for estimating the trajectory of a 
selected object based on an indicated position of the object at a first Instant, an 
indicated position of the object at a second instant, the time elapsed between said two 
instants, and physical assumptions of the object's trajectory (see column 2, line 3 and 
figure 6a-6d). 

1 1 . Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Moezzi 
et al. (US Patent No. 5,850,352) as applied to claim 10 above, and further in view of 
Maclnnis et al. (US Patent number 6,570, 579). 
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As to claim 1 1 , IVIoezzi disclose, keying process. However Moezzi does not 
specifically disclose chroma keying process or a difference keying process. Mclnnis 
from the same field of endeavor disclose said keying process is a chroma keying 
process (example, chroma keying, columns, lines 62). It would have been obvious to 
one of ordinary skilled in the art to modify keying process presented by Moezzi with the 
feature of chroma keying process as showmen by Mclnnis. The motivation to combine 
the two references determines whether each pixel is opaque or transparent based on 
the color of the pixel (see Maclnnis, column 15, lines 40-45). 

Conclusion 

12. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

The US Patent Number: 4, 956,706 by Akio Ohba, is cited to teach apparatus for 
processing image. 

The US Patent Number: 6,990,681 by Wang et al., is cited to teach enhancing 
broadcast of an event with synthetic scene using depth. 

The US Patent Number: 7,042,493 by Prandoni et al., is cited to teach 
Automated video sequences. 

The US Patent Number: 7,106,361 , by Kanade et al., is cited to teach system 
and method for manipulating the point of interest in sequence of images. 
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The US Pub No: 2003/0043270 by Rafey et aL, is cited to teach extracting a 
depth map from known camera and model tracking data. 

1 3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Meseker Takele whose telephone number is (571) 270- 
1653. The examiner can normally be reached on Monday - Friday 7:30AM- 5:00PM est. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Xiao Wo can be reached on (571) 272-2100, The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Infonmation regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. . 
For more infonmation about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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